Combined learning methods and mining complex data

Machine Learning has shown tremendous progress in the last decades. Started as a sub-domain within Artificial Intelligence it has grown into an independent field of study. New research problems have been identified, many new methods have been introduced and the number of their applications in various areas is still increasing. Machine Learning together with the related, younger field of Data Mining have become powerful tools for knowledge discovery in databases and intelligent data analysis. One of the most popular tasks in learning is supervised learning, in particular discovering classification knowledge from data, which usually leads to creation of classifiers. The classifiers could be constructed in different ways, the most common ones being decision trees, decision rules, Bayesian approaches, instance based learning, artificial neural networks, discriminant function or support vector machines. Nevertheless, most of both past and present research concerns developing single learning algorithms. However, according to both theoretical (e.g. “no free lunch theorem”) considerations as well as experimental experiences one cannot expect of one single algorithm to be simply the best. Each reasonable algorithm has its own area of superiority and it cannot outperform others in all possible learning problems. Overcoming limitations of single algorithms and improving predictive accuracy could be achieved by integrating several diversified classifiers into a combined system. Such systems, known under the names ensembles or multiple classifiers, have received a noticeable research and application interest at least since the late 80’s. As a result, several different approaches for generating component classifiers and aggregating their predictions have been proposed. The most influential works include Bagging and Boosting, which manipulate learning examples. However, solving difficult problems involves also changing feature representations and integrating classifiers learned from different feature subsets. Multiple classifiers are also used in the context of learning from dynamic data streams, where the target classes may change over time. Furthermore, for some problems hybrid classifiers are suitable, which use multi-strategic learning that fits the data with different representations. Another critical issue concerns observation that most of learning or data mining algorithms have been developed for stable, tabular representations of data. Such tabular data are present in many software systems and they are often easily obtained from relational databases – commonly met in any application domain. Moreover, data coming from other sources are often transformed into this format; see, e.g., processing text documents with natural language techniques or extracting numerical features from images. However, in some domains this data model appears to be too restrictive. Many modern automatic systems in science, engineering, medical or social fields are able to collect larger data with increasing their structure. This growing complexity comes both from the need for getting richer and more precise descriptions of real world objects and from development of new technologies for their measuring or collecting. For example, such complex data may appear in biology, bio-informatics, analysis of heterogeneous representations/files inside the patient electronic record, mining graph structure, industrial